Modelling LGMD2 visual neuron system

Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bioinspired nonlinear functions, are introduced in our model, to achieve LGMD2’s collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition

Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background

The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is filtered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: first, the neural network fulfilled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector

Towards a Dynamic Vision System - Computational Modelling of Insect Motion Sensitive Neural Systems

For motion perception, vision plays an irreplaceable role, which can extract more abundant useful movement features from an unpredictable dynamic environment compared to other sensing modalities. Nowadays, building a dynamic vision system for motion perception in a both reliable and efficient manner is still an open challenge. Millions of years of evolutionary development has provided, in nature, animals that possess robust vision systems capable of motion perception to deal with a variety of aspects of life. Insects, in particular, have a relatively smaller number of visual neurons compared to vertebrates and humans, but can still navigate smartly through visually cluttered and dynamic environments. Understanding the insects' visual processing pathways and methods thus are not only attractive to neural system modellers but also critical in providing effective solutions for future intelligent machines. Originated from biological researches in insect visual systems, this thesis investigates computational modelling of motion sensitive neural systems and potential applications to robotics. This proposes novel modelling of the locust and fly visual systems for sensing looming and translating stimuli. Specifically, the proposed models comprise collision selective neural networks of two lobula giant movement detectors (LGMD1 and LGMD2) in locusts, and translating sensitive neural networks of direction selective neurons (DSNs) in flies, as well as hybrid visual neural systems of their combinations. In all these proposed models, the functionality of ON and OFF pathways is highlighted, which separate visual processing into parallel computation. This works effectively to realise neural characteristics of both the LGMD1 and the LGMD2 in locusts and plays crucial roles in separating the different looming selectivity between the two visual neurons. Such a biologically plausible structure can also implement the fly DSNs for translational movements perception and guide fast motion tracking with a behavioural response to visual fixation. The effectiveness and flexibility of the proposed motion sensitive neural systems have been validated by systematic and comparative experiments ranging from off-line synthetic and real-world tests to on-line bio-robotic tests. The underlying characteristics and functionality of the locust LGMDs and the fly DSNs have been achieved by the proposed models. All the proposed visual models have been successfully realised on the embedded system in a vision-based ground mobile robot. The robot tests have verified the computational simplicity and efficiency of proposed bio-inspired methodologies, which hit at great potential of building neuromorphic sensors in autonomous machines for motion perception in a fast, reliable and low-energy manner

Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the first-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are first, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efficient visual sensor, and realizing the revealed specific characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot

Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network

How do animals, e.g. insects, detect meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? This open question has been very attractive for decades. In this paper, with respect to latest biological research progress made on motion detection circuitry, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely motion and position pathways explored in fly visual system, for mimicking the tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information on moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) Compared to conventional computer vision techniques, it turns out the computational simplicity of this model may benefit the utility in small robots for real time fixating. (2) The hybrid neural network structure fulfills the characteristics of a putative signal tuning map in physiology. (3) It also satisfies with a profound implication proposed by biologists: visual fixation behaviors could be simply tuned via only the position pathway; nevertheless, the motion-detecting pathway enhances the tracking precision

Improved Collision Perception Neuronal System Model with Adaptive Inhibition Mechanism and Evolutionary Learning

Accurate and timely perception of collision in highly variable environments is still a challenging problem for artificial visual systems. As a source of inspiration, the lobula giant movement detectors (LGMDs) in locust’s visual pathways have been studied intensively, and modelled as quick collision detectors against challenges from various scenarios including vehicles and robots. However, the state-of-the-art LGMD models have not achieved acceptable robustness to deal with more challenging scenarios like the various vehicle driving scenes, due to the lack of adaptive signal processing mechanisms. To address this problem, we propose an improved neuronal system model, called LGMD+, that is featured by novel modelling of spatiotemporal inhibition dynamics with biological plausibilities including 1) lateral inhibitionswithglobalbiasesdefinedbyavariantofGaussiandistribution,spatially,and2)anadaptivefeedforward inhibition mediation pathway, temporally. Accordingly, the LGMD+ performs more effectively to detect merely approaching objects threatening head-on collision risks by appropriately suppressing motion distractors caused by vibrations, near-miss or approaching stimuli with deviations from the centre view. Through evolutionary learning with a systematic dataset of various crash and non-collision driving scenarios, the LGMD+ shows improved robustness outperforming the previous related methods. After evolution, its computational simplicity, flexibility and robustness have also been well demonstrated by real-time experiments of autonomous micro-mobile robots

Complementary Visual Neuronal Systems Model for Collision Sensing

Inspired by insects’ visual brains, this paper presents original modelling of a complementary visual neuronal systems model for real-time and robust collision sensing. Two categories of wide-field motion sensitive neurons, i.e., the lobula giant movement detectors (LGMDs) in locusts and the lobula plate tangential cells (LPTCs) in flies, have been studied, intensively. The LGMDs have specific selectivity to approaching objects in depth that threaten collision; whilst the LPTCs are only sensitive to translating objects in horizontal and vertical directions. Though each has been modelled and applied in various visual scenes including robot scenarios, little has been done on investigating their complementary functionality and selectivity when functioning together. To fill this vacancy, we introduce a hybrid model combining two LGMDs (LGMD-1 and LGMD2) with horizontally (rightward and leftward) sensitive LPTCs (LPTC-R and LPTC-L) specialising in fast collision perception. With coordination and competition between different activated neurons, the proximity feature by frontal approaching stimuli can be largely sharpened up by suppressing translating and receding motions. The proposed method has been implemented ingroundmicro-mobile robots as embedded systems. The multi-robot experiments have demonstrated the effectiveness and robustness of the proposed model for frontal collision sensing, which outperforms previous single-type neuron computation methods against translating interference

Robustness of Bio-Inspired Visual Systems for Collision Prediction in Critical Robot Traffic

Collision prevention sets a major research and development obstacle for intelligent robots and vehicles. This paper investigates the robustness of two state-of-the-art neural network models inspired by the locust’s LGMD-1 and LGMD-2 visual pathways as fast and low-energy collision alert systems in critical scenarios. Although both the neural circuits have been studied and modelled intensively, their capability and robustness against real-time critical traffic scenarios where real-physical crashes will happen have never been systematically investigated due to difficulty and high price in replicating risky traffic with many crash occurrences. To close this gap, we apply a recently published robotic platform to test the LGMDs inspired visual systems in physical implementation of critical traffic scenarios at low cost and high flexibility. The proposed visual systems are applied as the only collision sensing modality in each micro-mobile robot to conduct avoidance by abrupt braking. The simulated traffic resembles on-road sections including the intersection and highway scenes wherein the roadmaps are rendered by coloured, artificial pheromones upon a wide LCD screen acting as the ground of an arena. The robots with light sensors at bottom can recognise the lanes and signals, tightly follow paths. The emphasis herein is laid on corroborating the robustness of LGMDs neural systems model in different dynamic robot scenes to timely alert potential crashes. This study well complements previous experimentation on such bio-inspired computations for collision prediction in more critical physical scenarios, and for the first time demonstrates the robustness of LGMDs inspired visual systems in critical traffic towards a reliable collision alert system under constrained computation power. This paper also exhibits a novel, tractable, and affordable robotic approach to evaluate online visual systems in dynamic scenes